Combining Optimization Methods Using an Adaptive Meta Optimizer

Optimization methods are of great importance for the efficient training neural networks. There many articles in literature that propose particular variants existing optimizers. In our article, we use combination two very different optimizers that, when used simultaneously, can exceed performance single problems. We a new optimizer called ATMO (AdapTive Meta Optimizers), which integrates simultaneously weighing contributions both. Rather than trying to improve each one, leverage both at same time, as meta-optimizer, by taking best have conducted several experiments on classification images and text documents, using various types deep models, demonstrated through proposed produces better